Appendix to Kernel Belief Propagation

Section 1 contains a review of Gaussian mixture BP and particle BP, as well as a detailed explanation of our strategy for learning edge potentials for these approaches from training data. Section 2 provides parameter settings and experiment details for particle BP and discrete BP, in the synthetic image denoising and depth reconstruction experiments. Section 3 contains a comparison of two different approximate feature sets: low rank approximation of the tensor features and low rank approximation of the individual features alone. Section 4 is an experiment on learning paper categories using citation networks. Sections 5 and 6 demonstrate the optimization objective of locally consistent BP updates, and provide a derivation of these updates in terms of the conditional expectation. Section 7 discusses the kernelization of Gaussian BP. Section 8 gives the error introduced by low rank approximation of the messages. We describe two competing approaches for nonparametric belief propagation: Gaussian mixture BP, originally known as non-parametric BP (Sudderth et al., 2003), and particle BP (Ihler & McAllester, 2009). For these algorithms, the edge potentials Ψ(x s , x t), self-potentials ψ(x t), and evidence potentials Ψ(x t , y t) must be provided in advance by the user. Thus, we begin by describing how the edge potentials in Section 2 of the main document may be learned from training data, but in a form applicable to these inference algorithms: we express P(x t |x s) as a mixture of Gaussians. We then describe the inference algorithms themselves. In learning the edge potentials, we turn to Sugiyama et al. (2010), who provide a least-squares estimate of a conditional density in the form of a mixture of Gaussians,